Accelerating Neural Network Inference With Processing-in-DRAM: From the Edge to the Cloud

Neural networks (NNs) are growing in importance and complexity. An NN’s performance (and energy efficiency) can be bound either by computation or memory resources. The processing-in-memory (PIM) paradigm, where is placed near within arrays, a viable solution to accelerate memory-bound NNs. However, PIM architectures vary form, different approaches lead tradeoffs. Our goal analyze, discuss, contrast dynamic random-access (DRAM)-based for NN efficiency. To do so, we analyze three state-of-the-art architectures: 1) UPMEM, which integrates processors DRAM arrays into single 2-D chip, 2) Mensa, 3-D-stacking-based architecture tailored edge devices, 3) SIMDRAM, uses the analog principles of execute bit-serial operations. analysis reveals that greatly benefits NNs: UPMEM provides 23× high-end graphics processing unit (GPU) when GPU requires oversubscription general matrix–vector multiplication kernel, Mensa improves efficiency throughput 3.0× 3.1× over baseline Edge tensor 24 Google models, SIMDRAM outperforms central unit/graphics 16.7×/1.4× binary We conclude ideal models depends on model's distinct attributes, due inherent architectural design choices.